A three-dimensional 3D NAND stacked memory device can be formed from an array of an alternating stack of insulating materials and spacer material layers that are formed as electrically conductive layer or replaced with electrically conductive layers. Memory openings are formed through the alternating stack, and are filled with memory stack structures, each of which includes a vertical stack of memory elements and a vertical semiconductor channel. A memory-level assembly including the alternating stack and the memory stack structures is formed over a substrate. The electrically conductive layers can function as word lines of a 3D NAND stacked memory device, and bit lines overlying an array of memory stack structures can be connected to drain-side ends of the vertical semiconductor channels.